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The Misalignment Effect Function (MEF) describes the decrement in manual performance associated with a rotation be- 
tween operators’ visual display frame of reference and that of their manual control. It now has been empirically deter- 
mined for rotation axes oblique to canonical body axes and is compared with the MEF previously measured for rotations 
about canonical axes. A targeting rule, called the Secant Rule , based on these earlier measurements is derived from a hy- 
pothetical process and shown to describe some of the data from three previous experiments. It explains the motion trajec- 
tories determined for rotations less than 65° in purely kinematic terms without the need to appeal to a mental rotation 
process. Further analysis of this rule in three dimensions applied to oblique rotation axes leads to a somewhat surprising 
expectation that the difficulty posed by rotational misalignment should get harder as the required movement is shorter. 
This prediction is confirmed. Geometry underlying this rule also suggests analytic extensions for predicting more gener- 
ally the difficulty of making movements in arbitrary directions subject to arbitrary misalignments. 


INTRODUCTION 

Though instrumentation, training or procedures can be used to 
manage the control difficulties posed by awkwardly oriented 
teleoperation cameras, users of remote systems need to be able to 
operate through rotated views of the worksites to monitor automa- 
tion. They also need to be prepared to take over in the event of 
failures. Indeed, in many cases the required instrumentation, sen- 
sors, and communication may simply not be present to enable the 
partial automation needed to provide users with intuitive geomet- 
ric relations between their control inputs and movement of the 
end effector they control 1 . Consequently, there is a need to under- 
stand the scientific basis of the control difficulties encountered 
when telerobot users are faced with a rotation between the control 
frame of their input devices and their remote view of the worksite. 
This rotation constitutes a misalignment of two three-dimensional 
frames of reference. We call the function relating the amount of 
misalignment to the decrease in user performance the Misalign- 
ment Effect Function (MEF) 2 . 

We define the MEF as the relative decrement in the efficiency of 
user interaction with objects under user spatial control due to a 
rotation between their viewing or display coordinates and their 
input control co-ordinates. This decrement can be measured in a 
number of ways, but we have chosen to use normalized Path 
Length (nPL) for theoretical reasons apparent below. Normalized 
Path Length is defined for any movement from a starting point to 
a target as the path length actually moved divided by the mini- 
mum (straight-line) distance from the start point to the target 
(Ellis, Yeom, & Adelstein, 2012). Because nPL can be objec- 
tively related to optimal performance for purely translational 
movements, this definition also provides a means for extension to 
other aspects of movement since for these extensions all that is 
needed is an alternative optimality criterion. The movement in 


See Macedo, Kaber, Endsley, Powanusom and Myung (1998) or design 
descriptions of the Intuitive Surgical Da Vinci telerobot for example of 
such partial automation. 

2 

Formerly called the Misalignment Disturbance Function (MDF). 


question then may simply be expressed as a proportion of the cor- 
responding optimal movement. 

In work reported last year (Ellis et al., 2012)) we have used nPL 
to measure the empirical MEF for pure pitch, roll or yaw rotations 
in an aeronautical coordinate system where yaw carries, pitch, 
carries roll with +x forward away from the upright user’s body. 
We demonstrated that pure roll rotations about the x-axis showed 
a distinctly more difficult pattern than pure pitch or yaw rotations 
about y and z respectively when results were integrated across all 
possible movement directions into the eight surrounding octants 
as described below. We argued that this difference may be due to 
the unique steering roles of pitch and yaw rotations versus roll for 
the lateral and vertical direction. Roll disturbs control in both the 
lateral and vertical directions while pitch or yaw only affect one 
of the two. This different performance under roll rotation awaits 
confirmation. 

We were also able to show that the MEF’s first — 1/3 of the MEF 
revealed a gradual loss of efficiency so that ultimately nPL ap- 
proximately doubles as rotations approach ~65. We proposed a 
purely geometric argument that can explain this part of the MEF. 

In the present paper we first review our previous results and de- 
rive our targeting rule in more detail. We then extend our meas- 
urement of the MEF in the experiment described below to rota- 
tions more representative of those actually encountered in the 
field in that they are not about canonical axes. In fact, the rota- 
tions that we have selected for the present experiment may seem 
more complicated than rotations about canonical axes because 
their descriptions in terms of Euler angles are not easily verbal- 
ized. We finally argue that such descriptive approaches focusing 
solely on the rotations themselves may need to be extended to 
include the user’s specific control rule to fully understand their 
control difficulty. 

Fig. 1 illustrates the basic movement task that we used. Partici- 
pants were placed in a simple virtual environment. They were 
instructed to move their hand to a central starting point within a 
large wire-frame sphere. Next, following the participant’s button 
press, a target would randomly appear some distance from the 


starting point. They were thereafter to manually move a cursor 
using their spatially-tracked right hand so as to make it touch the 
target, causing it to vanish. Participants would then return their 
hand to the starting point to prepare for the next target. 



Figure 1. This composited exocentric view shows a participant in our 


immersing virtual environment moving a computer-generated blue cur- 
sor ball to touch the larger, flat-shaded green, spherical target at the up- 
per left. Participants used a physical hand-held control to initiate tra- 
jectory recording and advancement to the next target (See Ellis et al, 

2012 for details). The yellow wire-frame sphere encircling the start point, 
shown for scale, disappeared on trial start. The two coordinate frames, 
not visible during testing, show a multi-axis rotational misalignment be- 
tween the display axes (lower left) and the control axes (upper left-cen- 
ter). 

As in our previous work, our focus is on users’ ability to over- 
come disturbances due to the unusual visual motion of their 
manually controlled cursor caused by experimentally imposed 
rotational misalignment. We believe the underlying control diffi- 
culty caused by control frame misalignment stems from the dis- 
turbed visual feedback that would otherwise help steer the user’s 
motion. Consequently, to exclude rotational cueing information 
from visually or kinesthetically sensed structures in our experi- 
mental studies, we restrict ourselves to spherical display elements. 
Ultimately, we seek to develop a general theory that will enable 
us to predict the underlying control difficulty for user movement 
toward an arbitrary target location subject to an arbitrary rota- 
tional misalignment. 

A Partial Theory of the MEF 

Fig. 2 summarizes some results from last year’s HFES Proceed- 
ings report in which the MEF was determined by pooling rota- 
tions about the canonical pitch, yaw and roll axes. The resulting 
function is usefully discussed in terms of the following three 
parts: the first slowly increasing part up to about yj = 65° of rota- 
tion, the second more rapidly increasing part leading to a peak 
somewhat greater than yj = 120°, and a third with a distinctive 
decrease coming down to a level at yj = 180° that is well above 
that for 0° rotation. A complete theory of the MEF must explain 
all of these features — features that can also be seen in other ear- 
lier work. (Fig. 2). We will focus only on the first part, below yr = 
65°, but will also comment in the Discussion section on rotations 
at and above yj = 90°. 


It is tempting to explain the MEF in terms of user internal infor- 
mation processing that might involve mental rotation. Such proc- 
essing may well be involved, especially in the presence of larger 
rotations. However, since the disturbance of user performance is 
inherently geometrical, it is informative to first look towards 
geometrical explanations that may involve simple targeting rules. 
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Figure 2. The MEF resulting from pooling results from Ellis, Yeom, & 
Adelstein (2012) for rotations about pitch, roll, and yaw for a task like 
that in this paper. For comparison, previously published results (Ellis & 
Adelstein, 2009) from a related 2D placement task are overlaid. Also 
overlaid are results from a 3D tracking task done with a perspective dis- 
play that subject to display-control coordinate misalignments (Ellis, Ty- 
ler, Kim & Stark, 1992). The three data sets show similar MEFs. 


In particular, we have examined a targeting rule that essentially is 
a definition of visual-motor coordination. It assumes human 
movements are made of submovements as posited by classic sam- 
ple-data control movement models (Stark, Iida & Willis, 1961). 
The rule is as follows: First, set the current kinesthetic direction 
for each submovement, Xi ? equal to visual direction at each sub- 
move-ment start, cf. Second, iterate across all submovements, 
making the kinesthetic direction equal to the estimated visual di- 
rection at each submovement until target contact (Fig. 3). Implicit 
in this rule is that completed movement paths consist of a se- 
quence of progressively refined submovements arising from the 
sample-data control at the heart of vertebrate kinesiology (e.g., 
Stark et al, 1961; Crossman & Goodeve, 1983). 

As shown for the planar case depicted in Fig. 3, integration across 
all of a movement’s constituent submovements yields a simple 
expression for the expected nPL. Since l/cos(y/) = sec(^), we call 
this expression the Secant Rule for coordinated hand movement. 

In fact, this targeting defines an equivalence class across velocity 
since a wide variety of velocity profiles will give rise to the same 
general trajectory form, though the piece-wise approximation to 
the equiangular spiral will vary. This equivalence class over a 
variety of velocity profiles is directly analogous to the equiva- 
lence class over ratios of movement distances to target widths 
defined by Fitts’s Law 3 . One can see in Fig. 2 that the targeting 
process provides a fairly good fit to the empirical MEF for rota- 


Deeper connections to Fitts’s Law exist showing that target width plays 
the role of a normalizing factor but they are out of this paper’ s scope. 


tions less than ~65°, and does so without free parameters! 
Clearly, this simple targeting must break down at some point 
since sec(^) op as y/ 90° where the targeting rule results in 
an orbit around the target. Therefore, the underlying targeting 
process must change beginning at or near this rotation angle. 


nPL = 1 Ixjl / dj and lxjl« dj 
so i Ixilcos(tp) ~ dj 
and nPL - l/cos(ip) 
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Piecewise approximation of 
an equiangular spiral. 


Figure 3. Iterative targeting geometry for the Secant Rule is sketched 
where intention i = Xi. Shaded (blue) indicates the plane of action. 



Figure 4. The targeting rule is visualized more generally in 3D space 
showing the instantaneous plane of action (light blue shaded triangle) 
determined by the intended direction of movement (light vectors) and the 
achieved direction of movement (darker vectors). 


cated in Figs. 3 and 5. This ratio is, in fact given by cos(y/), 
where y/ is the rotation angle. Data analyses suggest the critical 
value for this ratio to successfully predict performance is around 
0.56, which corresponds to cos(65°). Thus, 65° is approximately 
the largest rotation for which the Secant Rule applies. The critical 
value could be interpreted as the minimum acceptable efficiency 
factor with which the advance brings the operator’s controlled 
element towards their targets (Fig. 3, 4 and 5). Interestingly, this 
factor is reminiscent of the constant, A, used by Crossman and 
Goodeve (1983) in their derivation of Fitts’s Law, where A repre- 
sents the proportion of the remaining distance moved to the target 
during each submovement. This parallel suggests the devel- 
opment of a Fitts’s Law equivalent for off-axis movement. 



Figure 5. Enlargement of targeting rule visualization in 3-D space. The 
rotation, W, of the intended movement vector (lightest arrow) to the 
achieved movement vector (dark arrow) defines the blue plane of action. 

Curiosity regarding generalization of the MEF to rotations about 
noncanonical axes naturally arises and we have begun to inves- 
tigate more generalized conditions in the following experiment. In 
order to think about the possible effects of such rotations on the 
MEF we have extended the geometry of Fig. 3 to three dimen- 
sions in Fig. 4. This figure represents an arbitrary rotation associ- 
ated with an arbitrary motion from start point S to target 7j . The 
rotation axis is represented by the thick dark black arrow passing 
through S. The arc around the axis ((//shows the amount of rota- 
tion. The plane of rotation is represented by a disk centered at S. 
Any possible rotation may be represented as done in Fig. 4. 


The range of conditions over which of the proposed targeting rule 
applies is probably determined by an optimization process analo- 
gous to that suggested for raptors that move, i.e., fly, along equi- 
angular spirals when they dive onto their prey. They do so be- 
cause from their normal surveillance height they must target their 
prey using the higher-resolution of their two foveas, which is ro- 
tated laterally ~45° with respect to their body axis (Tucker, 2000). 
It appears the birds fly time-optimally by trading off the longer 
flight path needed to maintain use of this fovea against the added 
speed they can achieve by avoiding head turning that would slow 
them due to drag. Human telerobot operators may similarly trade 
off the time to move along an equiangular spiral against the added 
time and effort needed to determine a compensatory path. This 
tradeoff may be reasonably expected since arm control models are 
frequently time optimal. The operative ratio that may influence 
the human user’s tradeoff is the advance / intention ratio as indi- 


For rotations that we previously used about the canonical axes, all 
planes of rotation were ipso facto parallel to the canonical sur- 
faces orthogonal to the rotation, e.g., roll rotation about the x-axis 
has a plane of rotation parallel to the y-z plane. These rotation 
conditions are relatively easily described and understood by par- 
ticipants. But they are very special cases. A full theory of the 
MEF must extend beyond them to the situation shown in Fig. 4. 

Consideration of Fig. 4 as representing an arbitrary rotation leads 
to some qualitative conclusions from the geometry that we inves- 
tigate in the experiment below, deferring a more quantitative 
analysis for future reports. For example, one can immediately see 
that the specific position of the target with respect to the rotation 
axis has a major influence on the effect of the rotation: Targets 
located exactly on the axis, e.g., T 2 , are unaffected by the rotation. 
Those within plane of rotation it, are maximally affected, and 



correspond to the analysis in Fig. 3. Another interesting inference, 
however, is that targeting using the rule from Fig. 3 will result in 
a three-dimensional spiral path. The simulations and closed form 
equations describing this path will be treated in future reports. 
Significantly, for those rotations for which the Secant Rule ap- 
plies, the spirals are locally roughly planar. One may think of 
them as approximations of the planar spirals within a plane of 
action in that their length is closely predicted by the planar analy- 
sis used in Fig. 3. 

It is interesting to note that there is a great advantage for partici- 
pants who can stay in an initial plane of action shown in Figs. 4 
and 5. Staying in this plane turns their targeting task into a sim- 
pler one-de-'gree-of-freedom (dof) problem instead of a two-dof 
problem if they leave the plane. Accordingly, because displace- 
ment noise that takes the cursor out the plane of action has a 
much greater effect on the directional corrections needed to return 
to the plane as the cursor approaches the target, one might expect 
targeting to become more difficult for nearer targets than for far- 
ther ones. This effect is enhanced by the slowing of cursor motion 
as one gets closer to the target, which, in turn, makes visual dis- 
crimination of the error more difficult. We investigate this possi- 
ble effect of distance in the following experiment. 

METHODS 


Subjects: Ten unpaid volunteers (eight men, two women) aged 
24-65 participated. Subjects were screened for stereo vision, 
compatibility with the head-mounted display and provided IRB 
required consent for human experimentation. 


Head mounted display: A modified Rockwell- Collins SR80 bi- 
nocular head mounted displays was used. Descriptive details may 
be found in our previous paper (Ellis et al., 2012) and manufac- 
turer specifications 



Figure 6. Fick gimbal 
illustrating a yaw- 
pitch-roll Euler se- 
quence for one of the 
rotations used in the 
experiment. 


Figure 7. Approximate positions of the 28 tar- 
gets used for the two target distances shown by 
spherical red markers. Movements were from 
the center outward. Separate randomized se- 
quences of targets with a fixed rotation were 
used for blocks of targets for each participant. 


Computer hardware and software: The experiment was run in a 
custom virtual environment (VE) authored by Richard H. Jacoby, 
using lower-level tracking sensor interface software by AuSim, 

Inc. under Windows XP (SP2), rendering at 60 fps, with a meas- 
ured full system latency of ~30 ms. See (Ellis, et al., 2012). 


Experimental virtual environment: The VE created for the 
study was a simple room with dimensions roughly matching the 
physical room (4.0 x 4.5 x 2.9 m) in which the experiment was 


conducted. Diffuse lighting coming from virtual room sources 
mimicked the lighting in the real room. Texture maps that corre- 
sponded approximately to the room’s actual ceiling and floor 
provided some additional realism. 

Experimental design: A more detailed description of our gen- 
eral experimental procedure may be found in our previous paper 
(Ellis, Yeom, & Adelstein, 2012). It is important to reiterate here 
that the participants were told that when their task got harder, the 
goal of their movement was to try to move as they did when the 
task was easy during initial familiarization. We put no specific 
premium on quick motion onset, speed, or accuracy. 

Independent variables. 1) Target distances from the origin cen- 
tered in front of the participant at two levels were 1 1 cm or 22 
cm. Target locations are shown in Fig. 7. 2) Control-to-display 
coordinate rotations in terms of yaw, pitch, and roll angle were, 
respectively, (±45°, 35°, 0°), (±45°, 35°, ±30°), (±45°, 35°, 

±60°), and (±45°, 35°, ±90°). (See Fig. 6 for sample pose). Note 
that the signs for yaw and roll angles were matched for each Euler 
triple, yielding eight rotation levels. This sign matching resulted 
in symmetrical poses of the final Euler rotation axis 4 , e.g., (+45°, 
35°, +30°) is symmetric with the triple (-45°, +35°, -30°). Ulti- 
mately, we collapsed rotation across signs, as we found no reli- 
able asymmetries in the performance data. 

Dependent variables: Normalized Path length (nPL) was used as 
our principal dependent measure. It is a kind of efficiency 
measure, essentially a percent calculation that we infer also re- 
flects task difficulty on the presumption that participants are 
consistently striving for the same efficiency when they “try to 
move as they did when the task was easy.” 

RESULTS 

Fig. 8 plots the two significant main effects confirming the 
somewhat counterintuitive prediction that, at least in terms of 
nPL, movement towards the near targets is relatively longer and 
less efficient. There was no statistical interaction as is evident 
from the parallel traces of the two presented distance conditions. 
Note that the final 90° rotation in the Euler sequence (i.e., roll) 
has a much smaller impact on performance than a single 90° 
about a canonical axis shown in Fig. 2. 

DISCUSSION 

Because the present somewhat counterintuitive results are 
based on presumptive constant noise or jitter present in human 
motion, they emphasize that any full theory of the control diffi- 
culty introduced by a display-to-control rotation needs not only 
to predict the biases in errors but also the variances that are in- 
troduced. In this context we have observed in our data that 
variances in normalized Path Length for the first part of the 
MEF are a linear function of the introduced rotation. We plan 
to include this aspect of our data in future more complete tar- 
geting models as we lack adequate space in the present report 
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We refer to symmetry of the Euler axes. 
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Figure 8. Results for ANOVA F’s for skew-corrected log transformed 
data shown respectively. Performance ranged over about a factor of two 
with respect to optimal performance where nPL = 1 . 

A second observation that we can make is based on a comparison 
of performance under pure pitch, yaw, or roll rotations of 60° and 
90° from our prior study (Ellis et al., 2012) versus performance in 
the present experiment where roll (the third component of our 
Euler rotation sequence) of the same magnitude is instead applied 
on top of underlying yaw and pitch rotations. Despite the fact 
that the resulting poses in the present experiment are much more 
difficult to describe, our participants’ behavior was much less 
affected by these compound Euler rotation sequences than pure 
pitch, yaw, or roll. 

Our analysis of this surprising result is not complete. The princi- 
pal difference we have identified so far is that though the amount 
of twist about the rotation axis, (90.4°) is very close to that used 
for the canonical axes rotations of the previous experiment, the 
axis of this rotation has an azimuth of 140° and elevation of 5°. 
This direction places it far away from alignment with any canoni- 
cal axis and it may be a privileged direction with respect to the 
participants’ body. In future work we plan to examine explicitly 
the orientation of the rotation axes with respect to the partici- 
pant’s body to evaluate this factor systematically. 

A final comment regarding our Secant Rule is noteworthy in that 
it provides an explanation for a phenomenon reported by Abeele 
and Bock (2001). When testing transfer of sensorimotor adapta- 
tion to rotational misalignments, they noticed that participants 
who adapted to fixed rotations less than 90°, showed transfer to 
other rotations less then 90° but not to rotations greater than 90°. 
Conversely, those adapting to rotations greater than 90°, only 
showed transfer to other larger rotations. Our rule implies this 
pattern. Transfer from small to large rotations does not occur be- 
cause no amount of training will make the Secant Rule work for 
large rotations. Clearly, a different rule must be used for large 
and small rotations. 

In the end it may be necessary to appeal to “mental rotation” as an 
explanation for some aspects of the MEF, but we hope, those who 
take this approach, ourselves included, realize the violation of 


Occam’s Razor that such a tack involves. Appealing to a concept 
itself not well understood hardly promises an explanation that 
could have the ring of truth. In any case there are specific diffi- 
culties in any appeal to mental rotation. Careful measurement of 
the MEF generally shows that as the rotation angle increases past 
120° towards 180° there is a clear decrease in the function, which 
indicates that the task gets easier. This drop seems to be associ- 
ated with motor performance (Chintamani et al, 2010). Classic 
mental rotation functions, in contrast associated with perception 
or discrimination, continue to increase consistent with the idea 
that a more or less constant rate of mental rotation produces a 
continuous increase in the participants’ response times (Shepard 
& Metzler, 1971, but see Collishaw & Hole, 2002). 

It must be noted, however, that the decrease of the MEF for larger 
rotation angles may not be seen in studies without sufficient sta- 
tistical power or appropriate experimental design to reduce the 
significant noise associated with function measurements of large 
rotations. Indeed, any explanation of this later (i.e., >120°) part 
of the function also needs to explain this increased noise. Our 
expectation in this respect is that the noise arises from ambiguities 
and uncertainties as to the orientation of the error vector when 
submovement targeting errors are noticed. These ambiguities, 
which may amount to errors in understanding the local reference 
orientation of the plane of action , could result in large incorrect 
“corrective” movements out of the initial plane of action and 
could, by an argument similar to the one we’ve made about the 
target approach, also explain the rise in trajectory variability for 
the large rotations. 
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